The exploding popularity and sophistication of the internet has brought to bear easy access for anyone or any entity to publish, consume and aggregate content. Along with an explosion of content, the rate of appearance of advertisements that accompany content (and serve to monetize the content) is growing at a similar pace. Internet advertising supports a large and sophisticated ecosystem of participants including publishers, content providers, ad networks, ad agencies, ad aggregators, and ad arbitragers.
Within this ecosystem, about 70% of the advertising is served by third-party ecosystem of participants. Substantially all of the participants in the ecosystem desire to see steps taken to protect the user experience. Some techniques exist for monitoring performance and for reporting poor performance with the intent of reacting to poor performance (e.g. via policing actions or via corrective actions). In some cases, measured performance below a certain threshold drives business logic modules, which in turn are configured to recommend or implement such policing or remedial actions. In some cases such actions include automated real-time alerting components. During the performance monitoring, the aforementioned business logic modules collect measurements related to the advertisements (e.g. latency, click-through, quality etc). Some or all of the measurements can be used to score the advertisers, publishers, content providers, ad networks, ad agencies, ad aggregators, ad arbitragers, and other ecosystem participants. Such a score can be used to rank the participants relative to other participants, and such a ranking can be used by the advertising community at large to build reputations.
Some of the scoring criterion is quantitative in nature. Other scoring criterion is qualitative in nature. Yet, techniques are needed to reliably combine quantitative and qualitative characteristics of the advertising such that the score contribution from one score constituent (e.g. a quantitative measure) does not unfairly eclipse the score contribution from a different score constituent (e.g. a quality measure). As regards quantitative measurements, some of the participants are more technologically savvy than others, and understand the inner workings of an ad network. In contrast, some of the participants have more marketing savvy than others, but do not necessarily understand the inner workings of ad placement. Highly effective internet advertising demands high scoring from the network. (e.g. based on quantitative measures) as well as high scoring from the user population and/or from editorial staff (e.g. based on subjective, qualitative measures). However, in some situations, these measures may counteract one another. For example, qualitatively higher-scoring advertisements (e.g. using video, animations, pop-ups, dynamically assembled targeted ad content, etc) might demand ad network resources to such an extent that the user experience suffers (e.g. from low-scores in quantitative criterion).
Participants involved in delivering third-party advertising need to be ranked in order to protect the user experience. Although there exist techniques for scoring against quantitative criterion, and although there exist techniques for recognizing qualitative criterion, it remains to meaningfully combine scores from such quantitative and qualitative criterion. For this reason and other reasons, there exists a need for ranking vendors by combining quantitative and qualitative characteristics of third-party advertising.